CP-47655: [Toolstack] - Associate/disassociate VM to/from anti-affinity group

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…ffinity group

UT - CP-47304: [Toolstack] - Add data model for anti-affinity group

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…P-47304

CP-47304: Add anti-affinity groups to the datamodel

In VM anti-affinity datamodel PR (xapi-project#5573),
there is many-many relation between VMs and VM groups.
 When changing the VM.groups manually, the VM_group.VMs will be updated automatically.
But when removing a VM group, the VM.groups will not be updated automatically.
This commit aims to update VM.groups for all VMs in a VM group when it is being removed.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…A-391880

CA-391880: Update related field 'groups' of VM when destroying VM group.

VM in an anti-affinity group will choose a host based on the following steps:
1. If the VM is associated with an anti-affinity group, compute the number
   of VMs associated with this anti-affinity group for each host.
2. Group these hosts based on the VM's count. The hosts in one group have the
   same number of VMs in one anti-affinity group. Sort all the host groups
   ascending based on the VM's count.
3. If a host is the affinity host of this VM, it will be treated as a single
   host group and be put on the head of the group list.
4. Choose a host randomly which the VM can run on it from each group.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…P-47302

CP-47302: VM start with anti-affinity

Sync master into feature/vm-anti-affinity

When reverting to a snapshot, "groups" needs to be in `do_not_copy`
to reserve the latest value of the VM.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…A-392177

CA-392177: Keep current group after reverting from snapshot

Sync master into feature/vm-anti-affinity

Resolve dependency cycling issue if we call Xapi_alerts inside pool_features.
Error: dependency cycle between modules in _build/default/ocaml/xapi:
   Xha_statefile
-> Sm
-> Sm_exec
-> Xapi_session
-> Pool_features
-> Xapi_vm_group_helpers
-> Xapi_alert
-> Xapi_http
-> Xapi_session

Signed-off-by: Lunfan Zhang <Lunfan.Zhang@cloud.com>

The generation of breach alerts during operations on VMs or groups, triggered when all running VMs within a group are breached (i.e., placed on the same host).

The following may generate or dismiss alerts:

Starting and resuming a VM.
Stopping and suspending a VM.
Pausing and unpausing a VM.
Migrating a VM.
Moving a VM from one group to another.
Destroying a VM group.
Changing the license from Premium to Standard dismisses all breach alerts, while changing from Standard to Premium generates them.

In other scenarios, such as when a VM changes state from paused to shutdown, rebooting the VM will not trigger or dismiss alerts.

Signed-off-by: Lunfan Zhang <Lunfan.Zhang@cloud.com>

…official/CP-47656

CP-47656 VM anti-affinity generate breach alert

The recommendations are read from files '/etc/xapi.pool-recommendations.d/*'
and exposed to clients. They are read-only and only for clients, hence they
don't impact the behavior of Xapi directly but only indirectly as they
inform a client's behavior.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

The recommendations are read from '/etc/xapi.pool-recommendations.d/*.conf'.
When Xapi starts, it will read each file and update the recommendations into
the database. If we need to add recommendations to other type objects, e.g.
host, VM, etc in the future, just add a similar 'xapi.*-recommendations.d'
directory and define the path in xapi_globs.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…P-48570

CP-48570: Load recommendations from config file when Xapi starts

Check feature flag in these places:
1. VM start.
2. Host evacuation. When this PR is raised, the host evacuation PR is
   still in review. So this PR doesn't include the checking for host
   evacuation. It will be included in another new PR.
3. Create VM group.
4. VM.set_groups. Adding VMs to a group and removing VMs from a group
   are all forbidden. If customers need to remove VMs from a group,
   just destroy the group.
5. Send VM anti-affinity alerts.
Also, based on our discussion, the name of feature is changed from
`VM_anti_affinity` to `VM_group`.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…P-48011-review

CP-48011: XAPI Support anti-affinity feature check

Control domains, templates, and snapshots cannot be added to
VM groups.

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

…rom_master

Update feature/vm-anti-affinity by merging from master

Add func host_to_vm_count_map to be used, rename RefMap to HostMap

Signed-off-by: Gang Ji <gang.ji@citrix.com>

fixup review comments

Signed-off-by: Gang Ji <gang.ji@citrix.com>

Signed-off-by: Gang Ji <gang.ji@citrix.com>

…A-393421

CA-393421: Special VMs cannot be added to VM groups

Host evacuation plan with anti-affinity support will be carried out in 3
phases:

1. Try to get a "spread evenly" plan for anti-affinity VMs, done if
   all the rest VMs got planned using binpack, otherwise continue.
2. Try to get a "no breach" plan for anti-affinity VMs, done if all the
   rest VMs got planned using binpack, otherwise continue.
3. Carry out a binpack plan ignoring VM anti-affinity.

Signed-off-by: Gang Ji <gang.ji@citrix.com>

…eprecated

- Use cbt_enabled flag from plugin responses, to allow that a snapshot
  of a CBT-enabled VDI will itself be CBT-enabled.
- As part of vdi-data-destroy, record the VDI type as a custom key, so
  that on a subsequent sr-scan it still shows up as a metadata-only
  snapshot.

Signed-off-by: Robin Newton <robin.newton@cloud.com>

Hugo docs: remaining API docs and release info

Signed-off-by: Andrii Sultanov <andrii.sultanov@cloud.com>

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>

equivalents.

This allows support to be added to external_certificate_thumbprint_of_master for Sha1 fingerprints.

Signed-off-by: Steven Woods <steven.woods@citrix.com>

This code snippet was unchanged since the beginning,
but pylint complained:
```
R1720: Unnecessary "else" after "raise", remove the "else" and de-indent the code inside it (no-else-raise)
```

Signed-off-by: Edwin Török <edwin.torok@cloud.com>

Fixes: 500a1f7 ("XenAPI: suppress pytype false positives")

Signed-off-by: Edwin Török <edwin.torok@cloud.com>

CI: Complete parallel Coveralls uploads

…-49116

CP-49116: Add Sha1 support to external_certificate_thumbprint_of_master.

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

Signed-off-by: Bengang Yuan <bengang.yuan@cloud.com>

This reverts commit 31eeb93.

…ivate/asultanov/uri-improvement

Revert "CP-49953: Remove parse_uri, switch to using Uri module instead'"

…A-386173

CA-386173: Update the message of WLB authentication issue

…m-anti-affinity-to-master

Merge `feature/vm-anti-affinity` to `master`